Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-766] add unroll RNN for HybridBlock #11948

Merged
merged 11 commits into from
Feb 16, 2019
Merged

[MXNET-766] add unroll RNN for HybridBlock #11948

merged 11 commits into from
Feb 16, 2019

Conversation

zheng-da
Copy link
Contributor

@zheng-da zheng-da commented Jul 31, 2018 •
edited
Loading

Description

Currently, Gluon RNNCell unroll can't be used in Gluon HybridBlock. This unroll function works with an RNN cell and can be used in any HybridBlock.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@zheng-da zheng-da requested a review from szha as a code owner July 31, 2018 15:56
@zheng-da zheng-da changed the title [WIP] add unroll RNN for HybridBlock [MXNET-766] add unroll RNN for HybridBlock Aug 1, 2018
eric-haibin-lin
eric-haibin-lin reviewed Aug 4, 2018
View reviewed changes
Copy link
Member

@eric-haibin-lin eric-haibin-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

python/mxnet/gluon/contrib/rnn/rnn_cell.py Show resolved Hide resolved
@eric-haibin-lin
Copy link
Member

It will be great to have some perf results to decide if we want to use unroll with foreach for models in gluonnlp

@eric-haibin-lin
Copy link
Member

@szha could you also review?

@eric-haibin-lin eric-haibin-lin mentioned this pull request Aug 7, 2018
Open
@nswamy nswamy added Gluon RNN pr-awaiting-review PR is waiting for code review labels Aug 9, 2018
@lupesko
Copy link
Contributor

lupesko commented Aug 21, 2018

@szha bouncing again for your review.

szha
szha reviewed Aug 26, 2018
View reviewed changes
python/mxnet/gluon/contrib/rnn/rnn_cell.py Outdated


def _contrib_format_sequence(inputs, layout, in_layout=None):
assert inputs is not None, \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it be feasible to reuse rnn_cell._format_sequence?

szha
szha reviewed Aug 26, 2018
View reviewed changes
python/mxnet/gluon/contrib/rnn/rnn_cell.py Outdated


def unroll(cell, inputs, begin_state, drop_inputs=0, drop_outputs=0,
layout='TNC', valid_length=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This interface is not consistent with existing unrolls in that merge_outputs is not supported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that if merge_outputs is false, we have to know the length of the sequence.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest renaming

@lupesko
Copy link
Contributor

lupesko commented Sep 14, 2018

Looks like feedback addressed and ready for merge, @szha kindly review.

@vandanavk
Copy link
Contributor

@szha @sandeep-krishnamurthy for review

@zheng-da zheng-da closed this Sep 29, 2018
@zheng-da zheng-da deleted the unroll branch September 29, 2018 21:33
@zheng-da zheng-da restored the unroll branch September 29, 2018 22:37
@zheng-da zheng-da reopened this Sep 29, 2018
@sandeep-krishnamurthy
Copy link
Contributor

@zheng-da - What is the next step here?

@zheng-da
Copy link
Contributor Author

i don't think this PR will be merged. I'll close it.

@zheng-da zheng-da closed this Oct 11, 2018
@zheng-da zheng-da deleted the unroll branch December 9, 2018 01:41
@zheng-da zheng-da restored the unroll branch February 14, 2019 22:51
@zheng-da zheng-da reopened this Feb 14, 2019
eric-haibin-lin
eric-haibin-lin reviewed Feb 15, 2019
View reviewed changes
Copy link
Member

@eric-haibin-lin eric-haibin-lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you fix the conflict?

@eric-haibin-lin eric-haibin-lin merged commit 4b527b6 into apache:master Feb 16, 2019
stephenrawls pushed a commit to stephenrawls/incubator-mxnet that referenced this pull request Feb 16, 2019
@zheng-da @stephenrawls
[MXNET-766] add dynamic_unroll RNN for HybridBlock (apache#11948)
2c124f7 
* add contrib unroll.

* reenable some tests.

* fix a bug.

* fix lint.

* fix a bug.

* support diff layouts.

* update doc.

* use a diff default layout.

* remove _contrib_format_sequence.

* fix lint.

* rename.
jessr92 pushed a commit to jessr92/incubator-mxnet that referenced this pull request Feb 19, 2019
@zheng-da
[MXNET-766] add dynamic_unroll RNN for HybridBlock (apache#11948)
544419c 
* add contrib unroll.

* reenable some tests.

* fix a bug.

* fix lint.

* fix a bug.

* support diff layouts.

* update doc.

* use a diff default layout.

* remove _contrib_format_sequence.

* fix lint.

* rename.
drivanov pushed a commit to drivanov/incubator-mxnet that referenced this pull request Mar 4, 2019
@zheng-da @drivanov
[MXNET-766] add dynamic_unroll RNN for HybridBlock (apache#11948)
bd8db52 
* add contrib unroll.

* reenable some tests.

* fix a bug.

* fix lint.

* fix a bug.

* support diff layouts.

* update doc.

* use a diff default layout.

* remove _contrib_format_sequence.

* fix lint.

* rename.
vdantu pushed a commit to vdantu/incubator-mxnet that referenced this pull request Mar 31, 2019
@zheng-da @vdantu
[MXNET-766] add dynamic_unroll RNN for HybridBlock (apache#11948)
cb8156a 
* add contrib unroll.

* reenable some tests.

* fix a bug.

* fix lint.

* fix a bug.

* support diff layouts.

* update doc.

* use a diff default layout.

* remove _contrib_format_sequence.

* fix lint.

* rename.
haohuanw pushed a commit to haohuanw/incubator-mxnet that referenced this pull request Jun 23, 2019
@zheng-da
[MXNET-766] add dynamic_unroll RNN for HybridBlock (apache#11948)
bf98ad3 
* add contrib unroll.

* reenable some tests.

* fix a bug.

* fix lint.

* fix a bug.

* support diff layouts.

* update doc.

* use a diff default layout.

* remove _contrib_format_sequence.

* fix lint.

* rename.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@eric-haibin-lin eric-haibin-lin eric-haibin-lin left review comments

@szha szha szha left review comments

Assignees
No one assigned
Labels
Gluon pr-awaiting-review PR is waiting for code review RNN
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@zheng-da @eric-haibin-lin @lupesko @vandanavk @sandeep-krishnamurthy @szha @nswamy